The paper was to reduce biased estimation using new approach (Penalized maximum likelihood Estimation (PMLE) Method) in Logistic Regression. For this aim, unreal four small data sets were randomly generated. maximum likelihood Estimation (MLE) and PMLE Methods were applied and compared for separation case including biased estimation in Logistic Regression when one of the cells in 2 x 2 tables becomes equal to zero (separation problem). Parameters1 and their standard error obtained by using MLE for four data sets were 12.56±257.8, 13.46±264.3, 13.42±210.3, and 13.41±180.4, respectively, meaning that MLE’s are biased estimates. Corresponding values for PMLE method were found 2.28 ± 1.81, 3.05 ± 1.59, 3.45 ± 1.53, and 3.45 ± 1.53, respectively, meaning that PMLE’s was unbiased estimates. It is clear that standard error value for data set 1 reduced from 257.8 to 1.81 when using PMLE method for separation problem. According to PMLE Method, the odds of being coronary heart disease risk for smokers were increased 21.08 times than that for non-smokers smoking in data set 2, which is significant at 1% level. The odds of being coronary heart disease risk for smokers were increased 31.63 times than that for non-smokers in data set 3 (P < 0.001). The odds of being coronary heart disease risk for smokers were increased 41.93 times than that for non-smokers in data set 4. When one of the cells in 2 x 2 contingency tables becomes equal to zero, PMLE was more superior to MLE Method because PMLE Method may be performed unbiased (reliable) estimation.

The present work focuses on the second order Markov chain model which arises in a variety of settings and is well-suited to be modeled in many applications. The efficiency of the maximum quasi-likelihood estimators with the full maximum likelihood estimators for second order Markov chain models are given, besides the limiting normality results on the asymptotic properties of the associated estimates. Some efficiency calculations are also given to discuss the feasibility and computational complexity of the QL approach relative to the full likelihood approach.